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Abstract 

o . 

C^l . In this paper, the problem of distributed opportunistic channel access in wireless relaying is in- 

O 1 vestigated. A relay network with multiple source-destination pairs and multiple relays is considered. 

All the source nodes contend through a random access procedure. A winner source node may give up 
P_h ■ its transmission opportunity if its link quality is poor. In this research, we apply the optimal stopping 

' theory to analyze when a winner node should give up its transmission opportunity. By assuming the 

^vq , winner node has information of channel gains of links from itself to relays and from relays to its 

destination, the existence and uniqueness of an optimal stopping rule are rigorously proved. It is also 
found that the optimal stopping rule is a pure-threshold strategy. The case when the winner node does 
not have information of channel gains of links from relays to its destination is also studied. Two stopping 
problems exist, one in the main layer (for channel access of source nodes), and the other in the sub-layer 
(for channel access of relay nodes). An intuitive stopping rule, where the sub-layer and the main layer 
maximize their throughput respectively, is shown to be a semi-pure-threshold strategy. The intuitive 
stopping rule turns out to be non-optimal. An optimal stopping rule is then derived theoretically. Our 
research reveals that multi-user (including multi-source and multi-relay) diversity and time diversity can 
be fully utilized in a relay network by our proposed strategies. 
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I. Introduction 

In a wireless network, the wireless medium is shared by all users. The medium access control 
(MAC) layer is to coordinate the channel access of the users in a orderly and efficient manner. 
However, since the link between a transceiver pair usually experiences fading and/or shadowing, 
it is now well recognized that the MAC layer protocol should be jointly designed with the 
physical layer. This has led to a cross-layer design concept, namely, channel-aware scheduling 
or opportunistic channel access. The basic idea is to let the MAC layer aware of the physical 
layer states. If a transmitter finds that its channel quality is poor, it may give up or be asked 
to give up (by a central controller, such as the base station in a cellular network) its channel 
access opportunity, with an expectation that there are other users with good channel quality^ 

'This expectation is reasonable since the users' channels are independent. 
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and those users can utilize its channel access opportunity and gain more. Although the user may 
sacrifice in a short term, it will get more in the long term, because later the user, when with 
good channel condition, may utilize the channel access opportunity of other users that have poor 
channel quality. 

Opportunistic channel access has received much attention in the literature, particularly in 
centralized networks [1], [2]. A central controller can collect the channel quality information of 
the users, and schedule only those users with the best channel conditions. On the other hand, 
the research on distributed opportunistic channel access is still in its infancy. Without a central 
controller, it is hard for a user to decide when to give up its transmission opportunities. An 
intuitive way is to categorize the channel of a user into two states: good state when the channel 
gain is above a threshold; and bad state otherwise. Then a user gives up its channel access 
opportunity when its channel is bad. Apparently the multi-user diversity (i.e., different users 
experience different channel gains) and time diversity (i.e., a user experiences different channel 
gain when time varies) are not fully utilized by the intuitive method. This problem was address 
recently in [3], by means of optimal stopping. The major idea is to let all the users contend for 
channel access. It is found that, 1) if the winner in a contention has an achievable (transmission) 
rate smaller than a threshold (which can be obtained by solving an equation), it is optimal for 
the winner to give up its transmission opportunity and all users continue to contend again; and 
2) if the winner in a contention has an achievable rate larger than the threshold, it is optimal 
for the winner to stop here, i.e., to utilize the transmission opportunity and transmit its data. 
The beautiful part of the work is in the pure-threshold strategy, which is easy to implement. As 
extensions to the work in [3], interference channel which can tolerate multiple users transmitting 
is considered in [4] where more than one nodes can share the channel simultaneously, and delay 
constraints are considered for real-time service in [5]. Pure-threshold strategies are also derived 
in [4], [5]. 

In this paper, we investigate opportunistic channel access in a relay network, since wireless 
relaying has recently attracted a lot of research interests [6]— [16]. We focus on distributed 
opportunistic channel access in relay networks. We consider the scenario of multiple source- 
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destination pairs aided by multiple relays. Since each source-destination pair involves two hops: 
from source to relays and from relays to the destination, the problem of opportunistic channel 
access in relay networks is quite different from those in a single-hop network (e.g., in reference 
[3]-[5]), and is challenging as multi-source diversity, multi-relay diversity, and time diversity 
should be all exploited. Two scenarios are considered: when the a source node has information 
of channel gains of the links from itself to relays and from relays to its destination, and when a 
source node only has information of channel gains of the links from itself to relays. In the first 
scenario, it is found that a pure-threshold strategy uniquely exists to optimize the average system 
throughput. There are two stopping problems in the second scenario, one in the main layer (for 
channel access of source nodes) and the other in the sub-layer (for channel access of relay 
nodes). An intuitive strategy is proposed, which is shown to be a semi-pure-threshold strategy. 
However, the strategy is not optimal. We also theoretically derive an optimal strategy for the 
second scenario. The rest of this paper is organized as follows. System model is introduced in 
Section EL The system throughput maximization problem in the two scenarios are theoretically 
solved in Section III and IV, respectively, followed by concluding remarks in Section V. 

II. System Model 

Consider K source-destination pairs aided by L relays. Amplify-and-forward (AF) mode [7], 
[11], [17], [18] is adopted when a relay is helping a source-destination pair. The transmission 
power of a source node and a relay node is P s and P r , respectively. 

We use a similar random access procedure to that in [3]. In a slot with duration r, assume each 
source contends for channel access with probability p . Then in a slot, the contention is successful 
if there is only one contender, with probability p s = Kp (l — po) K ~ l . Therefore, the number of 
slots, K s , until a successful contention follows a geometric distribution with Prob(i^ s — n) — 
p s (l — p s ) n ~ l - Then the duration of total contentions until a successful contention is tK s with 
its expectation as — . 

If the ith source node wins a channel contention and transmits, and relay j helps forward its 
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traffic to the zth destination, the maximal rate that can be achieved in AF mode is 
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where and gji are channel gains from the ith source to the jth relay and from the jth relay 
to the ith destination. Assume the channel gains from a source to a relay and from a relay to 
a destination follow a complex Gaussian distribution with mean being zero and variance being 
aj and a 2 g , respectively. 

In this research we aim at distributed opportunistic channel access with the help of optimal 
stopping theory. Some notations are defined as follows. After the previous data transmission, 
an observation is the process of channel contention until a successful contention (i.e., a winner 
appears). After each observation, the winner decides whether to continue a new observation (i.e., 
a new contention round is started) or to stop observation (i.e., the user transmits its data). In 
the nth observation, let s(n) denote the contention winner, k(n) denote the time spent in the 
observation (equal to r times the number of slots used in the contention), / s ( n v (i = 1, ...,L) 
denote the channel gain between source node s(n) and the ith relay, and <fe( n ) denote the channel 
gain between the zth relay and destination node s(n). In other words, the observed information 
in the nth observation is: X(n) := {s(n), k(n), f s ( n )i(n), f s ( n )L{n), gi s ( n )(n), gL S (n)(n)}. 
For the nth observation, the reward y n is the total traffic volume that can be sent if the winner 
transmits its data traffic, which is a function of X(n), and the cost T n is the total waiting 
time from the first observation until the nth observation plus the data transmission time. If the 
procedure decides to stop at the iVth observation, then the average system throughput is p^. 
In the sequel, capital N is called the stopping time. And our objective is to find the optimal 
stopping time (also called optimal stopping rule), N*, which attains average system throughput 
sup -§7^4- Here E(-) means expectation. According to [19, Chapter 6], this maxim al-expected- 
return problem can be equivalently transformed into a standard form with its reward changed 
to be Yn — \*T N . In particular, to get N*, we need to find an optimal rule to reach maximal 
expected reward 

V*(\*) = E(Y N )-\*E(T N ) (2) 
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where A* satisfies sup {E(Yn) — X*E(Tn)} = 0. Here A* is actually the maximal system 

N>0 

throughput in our problem. This transformation method will be used when we solve the optimal 
stopping problems in our research, as shown in the subsequent sections. 

Assume that the winner in the nth observation (i.e., source node s(n)) has information of 
channel gains {f s ( n )i(n), ...J s{n)L (n), g ls{n) (n), g Ls(n) (n)}^ If the winner s(n) decides to 
stop, it selects the relay that renders the maximal source-to-destination rate, given as 

D r i ( ~\ \ P s P r\fs(n)j(n)\ 2 \g js{n )(n)\ 2 \ 

R n = max <Mog 2 l+ ' ; — — H . (3) 

je{l,...,L} I 1 + Ps\fs(n)j{n)\ 2 + Pr\gjs(n){n)\ 2 J 

The total transmission time is T, where the transmission time from source s(n) to the selected 
relay and from the selected relay to destination s(n) are both T/2. 

III. Average Throughput Maximization by Optimal Stopping Rule 
To formulate our research problem as an optimal stopping problem, in the nth observation, the 

n 

reward is Y n = \R n with the spent time denoted as T n = r Ki + T. For finding a strategy N* 

i=l 

to control source nodes' random access and finally achieve maximal average system throughput 
§7p4, it is equivalent [19] to design a rule which attains sup ( ^E(R N ) - X*E ( T + r J2 K i] 

( N> N>0 { \ i=l J 

where A* satisfies 

N 



sup { ^E(R N ) - X*E ( T + t V Ki )}=0. 



N>0 



i=l 

T 



\ i=l 

Before giving the optimal stopping rule N*, two conditions E |sup \^R n — AT + r J^j Kij | < 
oo and limsup <^R n — \ lT + T^2Ki\ > = — ooa.s. should be checked which guarantee the 

n-s-oo [ V i=l J J 

existence of an optimal stopping rule. Here A can be viewed as the system throughput, while A* 



has the physical meaning of maximal system throughput. 



sup(-|i? n — nc) 



< oo. 



Lemma 1: let c > 0, we have E(^R n ) < oo and E 
Proof: See Appendix I. 

By decomposition similar to (43) in [3] and using Lemma 1, the first condition for existence of 
an optimal stopping rule can be proved. 

2 The case when source node s(n) has only information {f a ( n )i( n ), fs(n)h{ n )} will be discussed in Section IV. 
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Lemma 2: The second condition is also satisfied, namely lim sup 

— oo a.s. 

Proof: See Appendix II. 

Based on Lemma 1 and Lemma 2, the existence of an optimal stopping rule is guaranteed. 

Theorem 1: The optimal stopping rule which achieves maximal system throughput sup -§^4 
is given as follows: N* = min {n > 1 : R n > 2A*} where A* is the solution of the equation 
E{mzx{£R n -\T,0}} = f a . 
Proof: See Appendix III. 

With threshold 2A* as a constant, our derived rule N* has a pure-threshold structure and achieves 
the maximal system throughput A* = j • 

Theorem 2: As the solution of the equation E {max {-^R n — AT, 0}} = — , the maximal 
system throughput A* always uniquely exists. 
Proof: See appendix rV. 

The uniqueness of A* is consistent with its physical meaning as the optimal system throughput. 

For the pure-threshold rule N* = min {n > 1 : R n > 2A*}, we can acquire following conclu- 
sion. 

Corollary 1: With {-R„} n =i,...,oo i-i.d. and pure-threshold structure of N*, the stopping time 
denoted N determined by the optimal stopping rule N* follows a geometric distribution with 
Prob(iV = n) = F R (2A*) n ~ 1 (1 — F R (2X*)) where F R means cumulative distribution function 
(CDF) of R n . Let R^* denote the achievable rate at the stopped time. It has the CDF as 
Fr n *( x ) = H x — ^^*) Fr i-f^(2\^) where /(•) means an indicator function. 

Corollary 2: With the stopping time N determined by the rule N* geometrically distributed, 
the expectation of the stopping time E(N) = 1 _ F ^ 2X *) i s finite. With E{rK n ) = according 
to wald theorem [19] we have E(T N ) = E(rK n )E(N) < oo. 

In fact, these results arise from the pure-threshold structure of the optimal stopping rule N*. 
In addition, the pure-threshold structure largely simplifies our strategy realization. In details, 
after the nth successful channel contention, source node s(n) wins the channel and calculates 
its achievable transmission rate R n (which is via the best relay). If R n > 2A*, source node s(n) 



transmits to the best relay node and the best relay node helps forward to destination node s(n); 
otherwise, source node s(n) gives up the transmission opportunity and re-contends the channel 
with the other K — 1 source nodes again. In this way, the maximal average system throughput 
A* can be achieved. 

IV. Case when a source node does not have information of channel gains in 

THE SECOND HOP 

In previous sections, the winner source node in each observation has the information of channel 
gains of the links from itself to relays and from relays to its destination. Next we consider a more 
practical case that the winner source node in each observation only has channel gain information 
of links from itself to relays. The channel access procedure includes two parts: from sources to 
relays, and from relays to destinations. In the first part of the channel access, in a slot (with 
duration r/2), each source node contends for channel access with probability p . Then in a slot, 
the contention is successful if there is only one contender, with probability p s = Kp (l — Po)^ -1 - 
Upon a successful contention at the nth observation, the winner (a source node), denoted s(n), 
decides whether to stop and transmit, or to give up its transmission opportunity and start a new 
contention with other source nodes. If the winner decides to stop, it broadcasts its data to all 
relays. In the broadcast, the channel gain information of its links to all relays is also included. 
Then the second part of channel access procedure starts. In a slot (with duration r/2), each relay 
node contends for channel access with probability p\. Then in a slot, the contention is successful 
if there is only one contender, with probability p r = Lp\{l— pi) L_1 . Upon a successful contention 
at the mth observation, the winner (a relay node), denoted s(m), decides whether to stop and 
transmit, or to give up its transmission opportunity and start a new contention with other relay 
nodes. If the winner decides to stop, it forwards its data (received from the source node s(n)) 
to the corresponding destination. 

In the first or second part of the channel access procedure, the number of slots spent for a 
successful source or relay contention is geometrically distributed, with expectation -if— or 

^Ps ^Pt 

respectively. 

It can be seen that the channel access is actually a bi-level stopping problem: the main layer for 

7 



channel access of source nodes, and the sub-layer for channel access of relay nodes. In the main 
layer, the source nodes do not have channel gain information of links in the second hop (from 
relays to destinations). However, it is assumed the source nodes have statistical information (e.g., 
channel gain distribution) of channel gains in the second hop. Therefore, in the main layer, the 
reward (which is the source-to-destination data volume) in the nth observation is the expected 
reward in the sub-layer. On the other hand, in the sub-layer, the channel gain information of 
links from the winner to relays is already known. In other words, the sub-layer stopping problem 
should be based on channel gain information of the links in the first hop. It is also assumed 
that, in the sub-layer, a winner in a successful contention can have its channel gain information 
to its destination. 

In the main layer, let n and N denote the observation index and stopping time, respectively. 
And in the sub-layer, let m and M denote the observation index and stopping time, respectively. 

A. Intuitive Stopping Rule 

An intuitive method to solve the bi-level stopping problem is to let the sub-layer and main layer 
apply optimal stopping theory to maximize sub-layer and main-layer throughput, respectively. 

In the sub-layer, the relays already know the information of {/ s (n)i( n )> ••■■> fs(n)L{ri)}. Then 
in the mth observation, the achievable rate of the winner, s(m), is 

R m = (log 2 (l + . (4) 



1 + P s \fs(n)s(m)(n)\ 2 + P r \g s ( m ) s (n){rn)\' 
And the reward in the mth observation is Y m = \R m . Then we need to find the the optimal 

stopping rule M* in the sub-layer to attain the maximal A* = sup %% M \ ■ 

AI>0 ( M> 

In the main layer, if the stopping time is N, then the reward is E(Y M *), and the waiting 
time is E(Tm*) + Tn- Then we need to find the optimal stopping rule N* to attain the max- 
imal sup „,%ffi Y ***yi Note that in the numerator or denominator in JP ^^ Yh !*}}, , , the inner 
expectation is for the second hop, while the outer expectation is for the first hop. 

For the sub-layer optimal stopping problem, we have the following theorem. 

Theorem 3: Conditioned on a main-layer channel gain realization {f s (n)i(n), f s (n)L(n)}, 
a sub-layer optimal stopping rule achieving the maximal sub-layer throughput A* = sup f 



is given as this form: M* = min{m > 1 : R m > A*} where A* is the unique solution of the 
equation E {max {R m — A, 0}} = and always exists. 
Proof: See appendix V. 



The sub-layer optimal stopping rule has the following property. 

Corollary 3: With channel gains {/ s ( n )i(n), f s ( n )L(n)} having finite values, < A* < oo, 
E(T M *) < oo and E(Y M *) < oo. 
Proof: See Appendix VI. 

Based on the acquired strategy M* for the sub-layer stopping problem, a main-layer optimal 
stopping rule which achieves maximal system throughput is given in the following theorem. 

Theorem 4: An optimal stopping rule for the main-layer problem is of the form N* = 
min [n > 1 : R l n - Y^l > 7*f } where 7* satisfies the equation E {max [R\ - - 7^, 0} } 

R l n = \*E(T M *) and R\ = E{T M *)l 
Proof: See appendix VII. Note that here 7* is actually the maximal main-layer system throughput. 

From Theorem 3 and 4 it can be seen that, the intuitive optimal stopping rule {N*, M*} with 
M* = min {m > 1 : R m > A*} and N* = min {n > 1 : R\ - ^*R\ > 7*f} has semi-pure- 
threshold structure. In details, with sub-layer stopping rule M*, its threshold is not a constant 
but a function of channel gains in the first hop. Different from M*, the main-layer stopping rule 
N* has a constant threshold. The intuitive stopping rule can be implemented as follows. 

For source node channel access, upon a successful contention, the winner s(n) has the 
information of its channel gains {/ s ( n )i(n), f s ( n )L(n)}. The winner can calculate R\ and 
R\ by solving the sub-layer optimal stopping problem conditioned on channel gains in the first 
hop as {/ s (T,)i(n), j s {n)L^p)\- For the source node channel access, 7* is a constant satisfying 
E{max{Ri- 1 Rl- 1 ^0}} = ^. 

. If R l n — j*Rn < 7*f , the source node gives up its transmission opportunity and re-contend 
with other source nodes. 

3 Note that here M* is the optimal stopping rule of the sub-layer conditioned on channel gains in the first hop as 
{/s(n)i( n )i ■■•! /s(n)L( n )}> an d ^* is me corresponding maximal throughput in the sub-layer stopping problem. 
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• If — 7*-R^ > 7*f-> the source node broadcasts its data to all relays, and the relay 
node channel access starts. Upon a successful contention of a relay node s(m) in the mth 
observation, for the particular channel gains in the first hop as {f s (n)i(n), f s ^ L (n)}, 
relay node s(m) calculates A* by solving E{max{R m — A,0}} = Then if R m < A*, 
the relay node s(m) gives up its transmission opportunity, and re-contend with other relay 
nodes; otherwise, the relay node s(m) forwards its received data (from source node s(n)) 
to destination node s(n), and the transmission process for the packet from s(n) is complete, 
and all source nodes start a new round of contention again. 

B. Non-optimality of Intuitive Stopping Rule 

The intuitive stopping rules {N*,M*} first maximizes sub-layer system throughput and then 
maximizes that of main-layer system. It is interesting to notice that the intuitive stopping rule 
is not optimal, as follows. 

The expected system throughput can be expressed as , } — — N * \ in the intuitive stop- 

E \ [ E \ T M* )\ N* + T N* ) 

ping rule, where the subscript N* in {\*E(T M *)] Nt and [E(T M *)] Nt means the optimal stopping 
rule in the sub-layer is conditioned on the channel gains in the first hop when the main layer 
follows the stopping rule N*. The sub-layer stopping rule M* maximizes X ^{T^i] ' • Considering 
the term TV* in the expression of the expected system throughput, the sub-layer stopping rule 
M*, which maximizes A ^j!^ M *? , may not maximize f^* e ( t m*)] n *} ^ p Qr exam p| ej f or a su j-,_ 

t '( 1 M*) E{[E(T M * )} N ,+T N * | 

layer non-optimal stopping rule M\ we have A^ < A*. However, if [E(T M \)] N * < [E(T m *)]n*, 
it is possible to have < rj^f% • 

C. Proposed Optimal Stopping Rule 

From the previous discussion, M* in the intuitive stopping rule maximizes the sub-layer 
system throughput, not the main layer system throughput. Therefore, in the proposed optimal 
stopping rule, we do not let the sub-layer maximize the sub-layer system throughput. Rather, we 
let the sub-layer achieve maximal average award sup E (Y M — lT M ), where 7 > represents 

M>0 

the throughput in the main layer. 
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T T 
max( -Rm~-l,W*(~f) 



Since we have two stopping problems, we use V* and W* to denote the maximal expected 
reward after the problem transformation (similar to (2)) in the main layer and sub-layer, respec- 
tively. 

For the sub-layer stopping problem, we have the following theorem. 

Theorem 5: For fixed 7 > 0, the optimal stopping rule M*{^) for maximizing E (Ym — 7?m) 
is of the form: M{^) = min {m > 1 : ^R m > W*(^j) + | 7 } where W*(j) satisfies the equa- 
tions: 

E 

Proof: See Appendix VIII. 

Although Theorem 5 is for any particular value of 7, it is desired the sub-layer stopping rule 
is corresponding to the maximal system throughput 7*. How to obtain the value of A* will be 
discussed shortly. Therefore, for the main-layer stopping problem, it is assumed that the sub-layer 
stopping problem follows the rule M*(j*), and we have the following theorem for the main-layer 
stopping problem, the objective of which is to achieve the maximal system throughput. 

Theorem 6: With the sub-layer system following the strategy M* (7*), an optimal strategy to 
maximize the average system throughput is given as N* = min {n > 1 : W*(j*) > |7*} where 
7* satisfies E [max(W*(r/) - f 7, 0)] = 
Proof: See Appendix IX. 

Based on Theorem 5 and 6, we can see that our proposed stopping rule {N*,M*} has the 
form of M( 7 *) = min {m > 1 : | R m > W*(Y) + f 7*} and N* = {n > 1 : W*(Y) > |t*}, 
which achieves average system throughput maximum 7*. Here 7* is a constant satisfying 



E 



max(Vr( 7 ) - ^7, 0) 



T7 



where W*("j) is unique root of E [max (^R m — ^7, W / *(7))] = W*(j) + Therefore, value 
of constant 7* can be obtained numerically. 

Note that the proposed rule {N*,M*} has also semi-pure-threshold structure, as in the main 
layer, the threshold is a constant, while in the sub-layer, the threshold W*{^*) is conditioned on 
the channel gain realization in the first hop. 
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The proposed stopping rule can be carried out as follows. Here we assume each node knows 
the value of constant 7*. 

For source node channel access, upon a successful contention, the winner s(n) has the 
information of its channel gains {/ s ( n )i(n), J s (n)i(n)}. The winner can calculate W*{^*) 
by solving the sub-layer optimal stopping problem conditioned on channel gains in the first hop 
as {/,(„)! (n), ...J s (n)L(n)}. 

• If W*(Y) < |-7*, the source node gives up its transmission opportunity and re-contend 
with other source nodes. 

• If W*(j*) > ?7*, the source node broadcasts its data to all relays, and the relay node 
channel access starts. Upon a successful contention of a relay node s(m), for the particular 
channel gains in the first hop as {/ s („)i(n), f s (n)L{n)}, relay node s(m) calculates 
W*(Y) by solving E [max (f R m - f 7*, W*(>y*))] = W*(Y) + g. If fiC < W*(Y) + 
^7*, the relay node s(m) gives up its transmission opportunity, and re-contend with other 
relay nodes; otherwise, the relay node s(m) forwards its received data (from source node 
s(n)) to the destination s(n), and the transmission process for the packet from s(n) is 
complete, and all source nodes start a new round of contention again. 

D. Optimality of Proposed Stopping Rule 

According to Lemma 6.1 and Theorem 6.1 in [19], we have the following properties of average 
reward V*(j). 

Lemma 3: For 7 > 0, the function V*(j) := sup {E{[E(Y M )] N - j{[E(T M )] N + T N }}} is 

7V>0 

decreasing and convex function of 7. 

Lemma 4: For some 7, V*( 7 ) := sup {E{[E(Y M )] N - 1 {[E(T M )] N + T N }}} = if and 

only ^ 1 - ^[ e{{[ 1 {Tm)]n J n}} 
Remark: By combing these two lemmas, for a fixed sub-layer stopping rule M we can find an 
optimal stopping rule which attains sup < r ^jJ^^^hU- . . I The rule A^^" is obtained by 

AT>0 t E \\1 E ( T m)\n+ t n}} J 

solving an optimal stopping problem with reward defined as [E(Y M )] N — 7^ {[E(T M )] N + T N } 
where 7^ = sup < — ? f.j}^^^} * 1. In other words, to prove the optimality of our proposed 

AT> { E \\i E ( T M)\ N +T N H J 
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stopping rule (N*, M*), we need only to prove that M* can maximize V*(7), as shown in the 
following lemma. 

Lemma 5: For 7 > 0, the function V*(y) := sup {E {[E(Y M )] N - 7 {[E(T M )] N + T N }}} 

N>0 

achieves its maximum by following the stopping rule M*. The rule M* is an optimal stopping 
rule providing maximal expected reward E (Ym — jTm) in the sub-layer. 
Proof: Since 

V*( 7 ): = sup{E{[E(Y M )} N - 1 {[E(T M )} N + T N }}} 

N>0 

= sup {E{[E(Y M ) -jE(T M )] N -lT N }} 

N>0 

for 7 > 0, by using the stopping rule M* to maximize the average reward E (Y M — 7T M ) in 
the sub-layer, V*( 7 ) can be maximized. ■ 
With sub-layer problem solved by M*, according to our remark of Lemma 3 and Lemma 4, an op- 
timal stopping rule N* attains maximal average system throughput 7* = sup < ^ r f.j}^} YM *^ N }, -■ -■ 

N>0 I E \\l E ( T M*)\ N + T N }} 

This completes the proof of optimality of our proposed stopping rule {N*,M*}. 

V. Conclusion 

In a wireless relay network, the source nodes and relays nodes all experience independent 

fading. It is desired to fully exploit the multi-source diversity, multi-relay diversity, and time 

diversity. To achieve this, opportunistic channel access is needed, which is investigated in our 

research in a distributed structure. For the two considered scenarios (with source nodes having 

or not having channel state information in the second hop), we derived the optimal rules for 

opportunistic channel access. This research should provide insight to the design of channel-aware 

MAC protocols in wireless relay network. Further research may include the cases with limited 

channel state information and with quality-of-service constraints. 

Appendix I 
Proof of lemma 1 

Achievable transmission rate at the nth observation and its expectation can be re-written as 
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where /(•) means an indicator function. 

Since and follow complex Gaussian distribution with mean being zero and variance 
being a 2 and a 2 , respectively, we have E(\fij\ 2 ) = 5j and E(\gji\ 2 ) = 5 2 . Then we have 

w = | {i E 1",'", M 1 + .5"gop )} 



(8) 



(«) K f f f 



In 2 



£1}} 



^- 2 LP s P r a 2 a 2 < oo 



where (a) comes from the fact that for x, y > 0, we have 

xy 



log 2 (l 



< l+s+y < X U 



(9) 



1 + x + y y ~~ In 2 ~~ In 2 
Based on [19, Theorem 4.1], from E(R n ) < oo, we have sup R n — nc) < oo a.s., which 

n 

leads to E |sup (f--R n — wc) j < oo. 



Appendix II 
Proof of lemma 2 



Using a similar method in [3], for < e < E(Ki) = —, we have the following decomposition: 



T 



R n -\(T + rJ2 K i 



i=l 



—R n - n\r | — -£ 

2 \Ps 



K 



(10) 



From [19, Theorem 4.1], - — e > 0, and the result E(R n ) < oo (as shown in proof of Lemma 

ps v 7 



1), we obtain that 



lim 

n— too 







—R n - n\r 









— oo a.s. 



(ID 



Next we focus on the second component on the right-hand side of (TTOl) . 



Using [19, Theorem 4.2], when E (± - e - K^\ < holds, E (sup J2 (~ ~ £ ~ K i) ) < 

' \n>0 i=l ' / 

< oo, where ^ — e — K^j = max(^ — e — K i7 0). 



oo if and only if E 



e - Ki 
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The first and second moments of ^ — e — Kj\ are: 
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6 - Ki 



e-K, 
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1 1 
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As a result, we have 
E 



limsupfjVi 

n^oo \ i=1 \Ps 



£-K, 



< E 



sup 



n>0 ~ \Ps 



e - K 



< oo 



which leads to 



lim sup 



±(l-e-K, 

^ V Ps 



i=l 



< oo a.s. 



(13) 



(14) 



(15) 



From (flOl) . (TTTT >. and (fl~5T) . we have lim sup 



-oo a.s. 



Appendix III 
Proof of theorem 1 

With samplings at each time observation i.i.d. with each other, a general optimal stopping rule 
givenin [19] is of the form = min {n > 1 : Y n > W*} where W* = E (V* +1 \X{1), ...,X(n)), 
Y n is a reward when stop at N = n and V* +1 represents expected reward following op- 
timal stopping rule which does not stop before time observation n + 1. Let V* represent 
the maximal average reward. With substitution the general form of iV* can be simplified as 

n 

N* = min {n > 1 : Y n > V* - C n } where Y n = ^R n - AT - Ar J2 Ki indicates the reward 

i=i 

n 

and C n = XtY^ Ki represents the cost of waiting time until n. Further simplifying it, N* 

i=i 

can be described as N* = min {n > 1 : \R n - AT > V*{\)} where N* is determined by 
A. To derive this rule, V* needs to be known. By the optimality equation of a general form 

V* = max {Y n ,E (V* +l \X(l),...,X(ri))}, we have: 

T 



n-l 



max 



i=l 



i=l 



i=l 



T 

V* = max <^ — R 



AT, V 



XrK n . 



(16) 
(17) 
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With expectations over two sides, we have: 

V*(X) = E jmax j|i? n - AT, I/*(A) J - Ar# n j . (18) 

By using [19, Theorem 6.1] if sup \%E{R N ) -XE[T + tJ2 KA \ = V*(X) = is achieved 
by a rule N* , it also achieves sup -§7^4- By setting V*(X*) = 0, we can derive A* as the solution 
of E {max {^R n — XT, 0}} = XrE(K n ) = —, which is the maximal system throughput. As a 
result, the optimal stopping rule turns out to be iV* = min{n > 1 : R n > 2X*}. 

Appendix IV 
Proof of theorem 2 



We have 



^jmaxj^i^- AT, 1 1 = J (x — XT)dFr 



where FT Rn (x) is the cumulative distribution function (CDF) of ^i? re . Therefore, we can say 
that E {max {^R n — A*T, 0}} is continuous and a decreasing function with respect to A. Also, 
— is a strictly increasing function with respect to A. Both functions are positive. Then it can 
seen that they two functions have one and only one intersection point in a two-dimensional plot 
with the horizontal axis being A. 

Appendix V 
Proof of theorem 3 

To guarantee the existence of the optimal stopping rule in the sub-layer problem, two sufficient 
conditions need to be proved. As a basis, we first prove the finiteness of E(R m ) where m is the 
index of observation in the sub-layer stopping problem. 

L 

E 

j=i 

- V^/lF/lncWl I P ° p r\fs{n)i( n )\ 2 \9js{n)(™)\ 2 \\\ 
~ Mi fi l^l 1 + l+P.|/. Wj (n)| s +iV| W .)Wpi/J 

(6) L 

< E {zTt2 P sPr\f s{ n )3 (n^E(\g Mn f)} 

3=1 

= t{-L^ P S P r\fsin)An)\ 2 <y 2 a }<00 

where (b) comes from ©. 
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«■) = « ?('(['W=i])M+^«*»}} 



With the finite property of E(R m ) (which is expectation over channel gains of the second hop), 
similar to the proofs of Lemma 1 and Lemma 2, the sufficient conditions of the optimal stopping 

m 

rule in the sub-layer can be proved. With the reward modified to Y m = ^R m — A^ — A| Ki> by 

i=l 

following the way in Appendix III we can obtain an optimal stopping rule for the sub-layer as the 
form: M* = min{m > 1 : R m > A*} where A* satisfies the equality £ {max — A,0}} = 
Similar to Appendix IV, the existence and uniqueness of A* can be easily derived. 

Appendix VI 
Proof of corollary 3 

With {f s (n)i( n ), fs(n)L( n )} sampled in the main-layer problem, we check properties of the 
roots A* of E {max {R m — A, 0}} = At first, we can find that E {max {i? m — A, 0}} is a 
decreasing function with respect to A and y~ is increasing function with respect to A. Hence, 
the uniqueness and non-negativeness of the root A* is guaranteed. 

Further, we have 



E{max{R m - A*,0}} 



< E{R m } (20) 

from 

< oo 

which leads to A* < oo. 

Stopping time M in the sub-layer is geometrically distributed and then according to wald 
theorem [19], E(T M *) is finite. With A* finite, we can also derive the finiteness of E{Y M *)- 

Appendix VII 
Proof of theorem 4 

In a similar way as the proof of Appendix III, we need to find an optimal stopping rule to 
maximize E |a*_E(T m .) — 7 (^E{T M *) + f + f 5^ K^j j, where 7 can be viewed as the system 
throughput. As functions of samplings 

{fs(n)l{n), f s (n)L(n)} 

at observation n in the main-layer problem, \*E(T M *) and E{T M *) can be respectively denoted 
as R\ and R 2 n . As a result, the problem changes to how to find an stopping rule N* which 
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satisfies 

^ 4-7(4 + 7T + iE^) [ = 8up^{4- 7 [4 + y + ^X)^ 



rp N* \ ■) ( ( / „ iV 



* * , I N>0 \ 11. 

1=1 /J — V K \ «=1 



where R\ > 0, R 2 n > and 7 > 0. Based on Lemma 2, it is clear that 



limsup \ R} n - 7 ( R 2 n + ^ + T - Ki \ \ = -00. 



2 2 

The other condition for optimal stopping rule's existence can be proved as follows: 



e 



E {sup jfli - 7 ^ + I + I g * J J J < £ {sup {«;, - uj T - (I 

* {■?> f 5 £ (£ " 6 " *) } } " ^ + * {t } (21) 

where < e < y. With finitely valued, we have E(Rl) < 00. By using Theorem 4.1 and 4.2 
in [19], we can prove finiteness of the first and second terms on the right-hand side of inequality 
(|2T) . Together with the fact that E jsup {-7^} j < °> we have 

£ jsup ji£ -7 ^ + I + ^ AiH J < 00. 

After proofs of sufficient conditions, we can find an optimal stopping rule N*. By the optimality 
equation, we obtain: 

n— 1 ( n n ^ 

i=l I t=l i=l J 



V* = max {i£ - 7 I - 7 <, 7* J _ 7 l Kn . 



(23) 



Similar to the way in Appendix III, we finally acquire the optimal stopping rule which maximizes 
the relay system throughput as the form N* = min {n > 1 : R\ — y*R„ > 7*f} where 7 * solves 
the equation £ {max {i?* - yR 2 n - 7^, 0} } = ^7. 

Appendix VIII 
Proof of theorem 5 

For fixed 7 > 0, we need to find an optimal stopping rule to maximize E (Y M — "fT M ) 
which is conditioned on a channel gain realization in the first hop. The sufficient conditions 
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for existence of an optimal stopping rule can be proved similar to Appendix VI. Different from 
the sub-layer in the intuitive stopping rule (which maximizes the sub-layer system through- 
put), here we maximize E (Ym — iTm), and thus have the optimal stopping rule: M{j) = 
min [m > 1 : \R m > W*(^) + ^7} where W*(^) satisfies the optimality equation 



E 



T T 
max( -i? m --7,W*(7) 



2p r 



Since E [max {^R m — ^7, W*(7))] is a continuously decreasing function and W*(^) + is 
an increasing function, a finitely unique intersection point VP* (7) should always exist. 

Appendix IX 
Proof of theorem 6 

Before deriving the main-layer optimal stopping rule N*, an extended version of Theorem 6.1 
in [19] should be given as a proof basic. That is, if sup E [YaKt) — = f° r fixed 7 we 

7V>0 

should have E [Y N {j) - jT N ] < which leads to < 7- 

With N* such that E [^(7) - lT N *} = 0, it can also achieve sup E Z^ { ] )] ■ With the finite 
property of W*(^y) which is a function of {s(n), /g( n )i(n), / s ( n )i( n )} proved before, we can 
get E [W*^)} < 00. In a similar way to Appendix I and II, the conditions which ensure the 
existence of the optimal stopping rule hold. Based on that, we can get the optimal stopping rule 
N(j) = [n > 1 : W*(^j) — ^7 > ^*(7)} where V*(j) satisfies the optimality equation as 



E 



max(W*(7)-|7,^(7)) 



2p s 



By setting V*^) = 0, the maximal system throughput 7* is the solution of 

E 

Therefore, the optimal stopping rule achieving 7* is of the form: N* = {n > 1 : W*^*) > f-7*}- 



wax{W*{-y) - I 7 , 0) 



T7 

2p~ s 
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